In today's technologically-driven society, users and businesses are increasingly relying on software application and computing systems for facilitating and providing various types of services. As the reliance and dependence on software applications and computing systems has increased, so has the need to provide rapid and accurate updates to software applications and computing systems that are tailored to users' and businesses' ever evolving needs. In order to ensure that rapid and accurate updates to software applications and computing systems are provided, efficient and effective testing of such software applications and computing systems should be performed to ensure that users and businesses experience minimal errors, minimal inaccuracies, and/or optimized functionality when interacting with such software applications and computing systems. To that end, in order to test, exercise, and/or navigate through a software application, it is often desirable to understand existing or desired constraints on fields (e.g., input fields) of the software application, the order of operations to be conducted within the software application, and/or functional transitions between various states of the software application.
Constraints may be restrictions on the valid values which may be entered into a particular field. For example, constraints may be physical, such as an English keyboard, which does not easily allow for non-English characters, such as Chinese. That is, a constraint constrains the user to the English character set. In certain instances, the constraint may be within the software, such as a constraint that does not allow a first user to update a second user's personal information. More commonly, a constraint can be a restriction on the allowable values that may be entered into some field on a computer application page. For example, maybe a “Month” field on some web page only allows the values 1-12, and any other value results in an error. In certain scenarios, the constraint may be that certain fields, menus, or buttons (or any other type of widget) are accessible (or not accessible) for a given user navigating the page. In certain embodiments, a field may comprise, but is not limited to, an input field (i.e. an input field of an application under evaluation 230 that can take any input), an output field (i.e. an output field of an application under evaluation 230 that can have any output), parameters (e.g. parameters of application programming interfaces (APIs)(e.g. REST APIs, HTTP/HTTPS calls, or other APIs)), an allowable value in an input and/or output media device (e.g. audio, video, image, text, virtual reality content, augmented reality content, any media of the first and/or second user device 102, 111), any type of field, or a combination thereof.
Typically, all constraints in some group may need to be satisfied to allow for some behavior, such as navigating to a next screen in a software application. Constraints may be derived or specified manually, or may be input via a template requirements entry process, such as an entry process facilitated by a computing system. Constraints may be on parameters in an API such as a REST API, HTTP/HTTPS API, or any other API. A computing system, such as the existing Rational Rose system, may be capable of generating a model of the software application, including constraints, if given a highly detailed and complete set of computer system requirements developed using a rigid grammar. Unfortunately, such computing systems do not significantly eliminate the additional manual labor involved in generating the desired model. Instead, such systems shift the manual effort from that of model development to one of detailed requirements development. While such systems may automatically generate tests from the manually developed model, the models must continually be updated and changed to reflect a rapidly evolving software solution. Such continual maintenance efforts are not cost effective, and consequently, tests generated through such manually specified processes are considered brittle, i.e., the tests cannot accommodate minor changes to the software or system under evaluation and break easily. In certain scenarios, constraints may be generated naively by utilizing brute force algorithms that try many different values. For example, when performing boundary detection, an algorithm used by such naive systems might be a binary search algorithm, which inputs successive values by dividing the search space at each stage in half. While in general such searches are computationally efficient when compared to a more simplistic solution which tests possible values in a linear sequence, binary search algorithms are significantly less efficient than guided systems which preferentially test high probability, predicted boundaries first. In certain scenarios, constraints may involve one or more parameters from one or more web pages, and may be written in relational logic, temporal logic, mathematical formulas, or many other types of grammars or languages.
Other systems may discover computer program constraints through the analysis of execution traces. In the context of websites, for example, in order to move past website pages and forms, current approaches may either require human aid for website form data selection or such approaches generate randomized data as inputs. Execution traces are then collected, by exercising an application utilizing this human aid. Given a suitable quantity of execution traces, currently disclosed systems may generate a set of likely program constraints. The process is error prone and difficult to verify, and requires a large volume of execution traces, making the approach impractical for extracting website semantics. Some research activities have demonstrated a limited capability to automatically explore a computer program and discover a generalized model of the computer program's behavior. However, such systems cannot effectively relate the self-discovered model to the external stories, requirements, or other data sources defining the desired functionality of the computer program. As a consequence, testing of the self-discovered model generates tautological results of limited utility. Such systems cannot discover discrepancies between the self-discovered model and the desired functionality of the computer program. As a consequence, such systems cannot produce tests that verify the desired functionality exists within the computer program. Other semi-automated testing, exercise, and navigation systems are dependent on the manual development or human directed semi-automated development of computer program tests or exercise and/or navigation tools. Such systems can incorporate the significant knowledge of the human operator, including the human's knowledge of information internal and external to the computer program, and consequently, such systems often include directed and efficient tests of computer program constraints. However, such system's dependence on human input significantly limits their application.
While testing systems currently exist for semi-autonomously testing software applications and computing systems, such systems cannot effectively associate available textual information from sources, from which constraints may be determined or inferred, with the input field and/or navigation tasks that the constraints may restrict. Additionally, even when such associations are available, currently existing testing systems fail to intelligently process the textual information from various sources to extract constraints for input values, order of operations, and/or functional transitions for a software application being evaluated. As a result, currently existing testing systems attempt to discover constraints or appropriate values for such constraints through inefficient trial and error, through the parsing of models constructed using extensive manual input and labor, by examining the underlying software of the software application in detail, and/or through manual training of various aspects of the software application and/or computing systems supporting the software application. As an example, a software application may have an input field named “Price”, which may have Minimum and Maximum subfields, currently existing testing systems are incapable of intuitively determining that the field is a dollar-denominated field. Such existing testing systems typically rely on trial and error to determine the correct data type and the constraints for the subfields. As another example, if an error message is outputted by a software application being tested, currently existing systems are unable to read and process the error message and to relate the error message to the constraints for the input fields of the software application. As a further example, a document may exist that describes the input field of the software application, however, currently existing systems are unable to associate the information in the document with the input field and/or determine constraints from the documentation. Notably, if the software application has many input fields and parameters, or even a few sufficiently complex parameters, it may be virtually impossible to satisfy all the constraints to allow for successful navigation through the software application using currently existing trial and error techniques because of combinatorial effects.
Based on the foregoing, for existing systems, substantial manual effort may be required to understand a software application and to build a model with states, transitions, and/or constraints. For large systems, such a manual effort is likely intractable, and, thus, businesses will continue with navigating, exercising, and testing of software applications by using solutions that are dependent on accessing software application source code, detailed software application models, and/or solutions requirement human written software scripts and/or tests. Further, as the model is constructed using manual techniques, which are themselves prone to error, significant additional manual validation and verification must be performed on the manually generated model before it can be used for effectively evaluating a target application. Additionally, for such systems, businesses may rely on human directed training (i.e. recording steps taken by a human operator), or the direct human exercise of the software application being evaluated.
As a result, current testing technologies and processes may be modified and improved so as to provide enhanced functionality and features. Such enhancements and improvements may effectively decrease the effort required to determine and test constraints, while simultaneously improving the accuracy of the testing system. Additionally, such enhancements and improvements may provide for optimized constraint determination, increased application navigation capabilities, increased testing capabilities, increased learning capabilities, increased autonomy, improved interactions with users or devices, improved user satisfaction, increased efficiencies, increased access to meaningful data, substantially-improved decision-making abilities, and increased ease-of-use. Furthermore, such enhancements and improvements may reduce processor, memory, and network bandwidth usage. Moreover, such enhancements and improvements may increase a testing system's ability to test software applications more efficiently and effectively.